<html>
<head><meta charset="utf-8"><title>mmap concerns · t-lang/wg-unsafe-code-guidelines · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/index.html">t-lang/wg-unsafe-code-guidelines</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html">mmap concerns</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="161854458"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161854458" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161854458">(Mar 27 2019 at 12:36)</a>:</h4>
<p>We are having a good discussion over in <a href="#narrow/stream/187831-t-compiler.2Fwg-self-profile/topic/mmap.20concerns" title="#narrow/stream/187831-t-compiler.2Fwg-self-profile/topic/mmap.20concerns">https://rust-lang.zulipchat.com/#narrow/stream/187831-t-compiler.2Fwg-self-profile/topic/mmap.20concerns</a>, but I figured this WG might have something to add.</p>
<p>TL;DR, I think that <em>any</em> usage of file-backed-mmap in Rust leads to UB. This is because we get a <code>*mut T</code> from mmap. If we convert it to a <code>&amp;T</code> or <code>&amp;mut T</code>, the rules of references could be broken by any arbitrary process writing to the backing file. This would be reflected in the Rust process, effectively causing an immutable reference to mutate or a mutable reference to mutate by "not us".</p>
<p>I'd love to know that my concerns are unwarranted!</p>



<a name="161856682"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161856682" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161856682">(Mar 27 2019 at 13:04)</a>:</h4>
<p><span class="user-mention" data-user-id="124289">@rkruppe</span> did point out that we might be able to get a <code>&amp;[Cell&lt;...&gt;]</code> (or equivalent) from mmap, which might sidestep all concerns.</p>



<a name="161857072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161857072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Hanna Kruppe <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161857072">(Mar 27 2019 at 13:09)</a>:</h4>
<p>Well, all the mutable-xor-shared concerns. There is also the question of how concurrent modification <em>by other processes</em> interacts with the memory consistency model, in particular whether volatile or atomic accesses are needed.</p>



<a name="161859514"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161859514" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161859514">(Mar 27 2019 at 13:39)</a>:</h4>
<p>These concerns are possibly better considered from the standpoint of shared memory and we at Standard thought a lot about it. The most trivial example would be two distinct virtual memory pages backed by the same physical memory page.</p>



<a name="161859560"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161859560" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161859560">(Mar 27 2019 at 13:39)</a>:</h4>
<p>basically the conclusion we arrived at is that the mmaped shared memory shall only be accessed via raw pointers.</p>



<a name="161859580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161859580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161859580">(Mar 27 2019 at 13:40)</a>:</h4>
<p>For now, anyway.</p>



<a name="161859669"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161859669" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161859669">(Mar 27 2019 at 13:41)</a>:</h4>
<p>We also considered using <code>&amp;[{Unsafe,}Cell&lt;_&gt;]</code> and friends, but we found that to be too inconvenient for other reasons (we want to be able to do atomic operations on the memory...)</p>



<a name="161870189"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161870189" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161870189">(Mar 27 2019 at 15:23)</a>:</h4>
<p>The problem with mmap is that two consecutive reads can return different values in a sequential program. Because of this, if the reads are not volatile, AFAICT the behavior is undefined. Whether you are reading from mmap'ed memory that's modified by different processes, whether the same physical memory page is mapped to two different virtual memory pages within the same process, or whether you are reading from a register that's modified by a sensor, does not really matter.</p>



<a name="161870573"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161870573" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161870573">(Mar 27 2019 at 15:26)</a>:</h4>
<p>Somebody made the argument that multiple processes can be considered multiple threads of execution, and therefore atomic read / stores are enough, but that's IMO a long shot. Multiple processes != multiple threads, and unless our memory model ends up being multi-process (which is something that AFAICT no practical PL memory model does), atomic reads / writes guarantees don't apply here. Synchronization across threads does not matter if the program is sequential, and a "sufficiently smart" compiler could prove that the program is single threaded and turn all atomic read / writes into normal read / writes.</p>



<a name="161870953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161870953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Gilcher <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161870953">(Mar 27 2019 at 15:30)</a>:</h4>
<p>Isn't that basically a similar problem to what the embedded working group approaches with libs like volatile cell? <a href="https://github.com/japaric/vcell" target="_blank" title="https://github.com/japaric/vcell">https://github.com/japaric/vcell</a> ?</p>



<a name="161875387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161875387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161875387">(Mar 27 2019 at 16:16)</a>:</h4>
<p>Yep, that's pretty much it.</p>



<a name="161875605"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161875605" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161875605">(Mar 27 2019 at 16:18)</a>:</h4>
<p>There is also: <a href="https://docs.rs/voladdress/0.2.3/voladdress/" target="_blank" title="https://docs.rs/voladdress/0.2.3/voladdress/">https://docs.rs/voladdress/0.2.3/voladdress/</a> which lets you handle slices as well</p>



<a name="161875740"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161875740" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161875740">(Mar 27 2019 at 16:20)</a>:</h4>
<p>note how weird the API of <code>VolBlock</code> there has to be</p>



<a name="161875787"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161875787" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161875787">(Mar 27 2019 at 16:21)</a>:</h4>
<p>basically when indexing into an element of a volatile array to get a reference, you need a volatile reference that performs volatile load / stores on access</p>



<a name="161884542"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161884542" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161884542">(Mar 27 2019 at 17:59)</a>:</h4>
<blockquote>
<p>Isn't that basically a similar problem to what the embedded working group approaches with libs like volatile cell? <a href="https://github.com/japaric/vcell" target="_blank" title="https://github.com/japaric/vcell">https://github.com/japaric/vcell</a> ?</p>
</blockquote>
<p>I'm glad I'm not the only one who had this thought (and that it appears to be valid)</p>



<a name="161884675"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161884675" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161884675">(Mar 27 2019 at 18:01)</a>:</h4>
<blockquote>
<p>two consecutive reads can return different values</p>
</blockquote>
<p>One argument made in the other thread was that in this case, data is never read via mmap. It's write-once-read-never. Does that change the situation?</p>



<a name="161930970"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161930970" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161930970">(Mar 28 2019 at 07:55)</a>:</h4>
<p>Well the problem is that your program behaves differently depending on how its optimized</p>



<a name="161931036"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161931036" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161931036">(Mar 28 2019 at 07:56)</a>:</h4>
<p>If one does not care about that, then one does not really care about UB I guess</p>



<a name="161951725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161951725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161951725">(Mar 28 2019 at 13:30)</a>:</h4>
<p><span class="user-mention" data-user-id="116155">@Jake Goulding</span> there's been a long discussion about mmap on a forum... last year or so. let me find that.</p>



<a name="161951846"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161951846" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161951846">(Mar 28 2019 at 13:32)</a>:</h4>
<p>here you go: <a href="https://users.rust-lang.org/t/how-unsafe-is-mmap/19635" target="_blank" title="https://users.rust-lang.org/t/how-unsafe-is-mmap/19635">https://users.rust-lang.org/t/how-unsafe-is-mmap/19635</a></p>



<a name="161951952"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161951952" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161951952">(Mar 28 2019 at 13:33)</a>:</h4>
<p>given that no H/W IO stuff is involved, I think that atomics and not volatile are the right tool here</p>



<a name="161951988"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161951988" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161951988">(Mar 28 2019 at 13:33)</a>:</h4>
<p>notice that "two consecutive reads can return different values" is an argument for atomic accesses just as much as it is one for volatile accesses</p>



<a name="161952111"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161952111" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161952111">(Mar 28 2019 at 13:34)</a>:</h4>
<p>mmap can share memory with threads that run in a different address space (aka process), but I dont see why that would make a fundamental difference compared to threads running in the same address space -- the relevant part is that the physical address spaces backing the virtual pages overlap.</p>



<a name="161952214"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161952214" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161952214">(Mar 28 2019 at 13:35)</a>:</h4>
<p>also <span class="user-mention" data-user-id="116155">@Jake Goulding</span> one terminology nit:</p>
<blockquote>
<p>any usage of file-backed-mmap in Rust leads to UB</p>
</blockquote>
<p>it is UB <em>only if the file gets mutated</em>. you make it sound like just the act of mmap'ing and taking a reference leads to UB, that is not true. (sorry if I misunderstood you.)</p>



<a name="161952292"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161952292" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161952292">(Mar 28 2019 at 13:36)</a>:</h4>
<blockquote>
<p>It's write-once-read-never. Does that change the situation?</p>
</blockquote>
<p>no, it just makes it harder to come up with examples.^^ "other parties can observe when the write happens" also means we have to use atomics/volatile.</p>



<a name="161965768"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161965768" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161965768">(Mar 28 2019 at 15:51)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> I would like to hear your argument about why atomics are not enough for when HW I/O is involved (or where you suggesting that they are?), you can think of HW I/O as some process with its own threads, on a separate CPU, writing to some memory location that your process is able to read.</p>



<a name="161965898"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161965898" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161965898">(Mar 28 2019 at 15:53)</a>:</h4>
<p>(deleted)</p>



<a name="161975290"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161975290" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161975290">(Mar 28 2019 at 17:38)</a>:</h4>
<p>AFAIK the difference between volatile and atomics is that volatile guarantees that the accesses will happen exactly in the order specified in the program, while atomics guarantees that your program will behave "as if" these accessed happened and "as if" they happened in that order - the difference between "exactly" and "as if" meaning that the compiler can eliminate consecutive read/writes for atomics (or dead ones) but not for volatile</p>



<a name="161975492"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161975492" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161975492">(Mar 28 2019 at 17:40)</a>:</h4>
<p>for memory mapped I/O whether the difference between "exactly" and "as if" matters is subtle - consider:</p>
<div class="codehilite"><pre><span></span><span class="kd">let</span><span class="w"> </span><span class="n">x</span>: <span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">T</span><span class="p">;</span><span class="w"></span>
<span class="kd">let</span><span class="w"> </span><span class="n">y</span>: <span class="o">*</span><span class="k">mut</span><span class="w"> </span><span class="n">T</span><span class="p">;</span><span class="w"></span>
<span class="n">assert</span><span class="o">!</span><span class="p">(</span><span class="n">x</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="n">y</span><span class="p">);</span><span class="w"></span>
<span class="kd">let</span><span class="w"> </span><span class="n">y_val</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">y</span><span class="p">.</span><span class="n">read</span><span class="p">();</span><span class="w"></span>
<span class="n">assert</span><span class="o">!</span><span class="p">(</span><span class="n">y_val</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="mi">42</span><span class="p">);</span><span class="w"></span>
<span class="n">x</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="mi">42</span><span class="p">);</span><span class="w"></span>
<span class="k">if</span><span class="w"> </span><span class="n">y</span><span class="p">.</span><span class="n">read</span><span class="p">()</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="mi">42</span><span class="w"> </span><span class="p">{</span><span class="w">  </span><span class="cm">/* dead code ? */</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
</pre></div>



<a name="161975565"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161975565" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161975565">(Mar 28 2019 at 17:41)</a>:</h4>
<p>with atomics, the two reads to <code>y</code> could be coalesced, and the <code>if</code> optimized away</p>



<a name="161975672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161975672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161975672">(Mar 28 2019 at 17:42)</a>:</h4>
<p>but while <code>x</code> and <code>y</code> point to two different virtual addresses,  they can refer to the same physical memory even within the same process  - if the reads are volatile, the <code>if</code> won't be removed, and its block will be executed when this is the case</p>



<a name="161976646"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161976646" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161976646">(Mar 28 2019 at 17:53)</a>:</h4>
<p>for this code to be correct using atomics, the <code>y.read()</code>s cannot be coalesced into one, that at least rules out using relaxed ordering</p>



<a name="161978282"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161978282" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161978282">(Mar 28 2019 at 18:11)</a>:</h4>
<p>but yeah, I think I see now that atomics (with the appropriate ordering) should be sufficient</p>



<a name="161983652"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/161983652" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#161983652">(Mar 28 2019 at 19:14)</a>:</h4>
<blockquote>
<p>it is UB <em>only if the file gets mutated</em>. you make it sound like just the act of mmap'ing and taking a reference leads to UB, that is not true. (sorry if I misunderstood you.)</p>
</blockquote>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> you understood me correctly. I was under the impression that Rust was taking the position of "if any code flow path could cause UB, it is always UB".</p>



<a name="162029923"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162029923" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162029923">(Mar 29 2019 at 09:21)</a>:</h4>
<p><span class="user-mention" data-user-id="132920">@gnzlbg</span> </p>
<blockquote>
<p>I would like to hear your argument about why atomics are not enough for when HW I/O is involved (or where you suggesting that they are?), you can think of HW I/O as some process with its own threads, on a separate CPU, writing to some memory location that your process is able to read.</p>
</blockquote>
<p>Atomics do not guarantee that every read/write hits the memory bus verbatim. For example, the compiler can optimize <code>x.store(5, some_ordering); let val = x.load(some_other_ordering)</code> assuming that <code>val == 5</code>. Atomics still assume that the observer is another thread running in the same memory model.  In fact if the compiler can prove that <code>x</code> is never shared with another thread, it can even downgrade the atomic accesses to non-atomic accesses.<br>
<code>volatile</code> OTOH is for the case where the observer basically literally sits on your memory bus and can observe every single read and write. It seems intuitively like that is a stronger observer and hence it should be legal to replace (relaxed!) atomic accesses with <code>volatile</code>, but that is not how memory models are defined. It might be possible to define them like that, but I know of no research looking into this. (and for non-relaxed atomic accesses this cannot work because of the synchronization they induce.)</p>



<a name="162030063"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030063" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030063">(Mar 29 2019 at 09:23)</a>:</h4>
<p>(This is basically what you said about "exactly" vs "as if", but IMO this answers the question about why atomics are not good enough for memory-mapped IO)</p>



<a name="162030143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030143">(Mar 29 2019 at 09:24)</a>:</h4>
<p>The question is then for mmapped files for interprocess communication (or virtual memory shenanigans), which atomic orderings are necessary ?</p>



<a name="162030210"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030210" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030210">(Mar 29 2019 at 09:25)</a>:</h4>
<blockquote>
<p>for this code to be correct using atomics, the <code>y.read()</code>s cannot be coalesced into one, that at least rules out using relaxed ordering</p>
</blockquote>
<p>I am afraid the area of virtual memory effects is entirely unspecified, so I dont think any standard literally says anything about your program. Anything we can say here is wild speculation.<br>
Any formal model I am aware of would allow the compiler to assume that objects at distinct virtual addresses are independent of each other.</p>



<a name="162030301"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030301" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030301">(Mar 29 2019 at 09:26)</a>:</h4>
<p>in the same way that the compiler can assume that another process won't mutate memory that you are using behind your back ?</p>



<a name="162030311"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030311" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030311">(Mar 29 2019 at 09:27)</a>:</h4>
<p>I suppose that is comparable, yes</p>



<a name="162030329"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030329" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030329">(Mar 29 2019 at 09:27)</a>:</h4>
<p>if you start messing with <code>/dev/mem</code>, you asked for it. basically. ;)</p>



<a name="162030333"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030333" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030333">(Mar 29 2019 at 09:27)</a>:</h4>
<p>unless you use volatile</p>



<a name="162030337"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030337" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030337">(Mar 29 2019 at 09:27)</a>:</h4>
<blockquote>
<p>The question is then for mmapped files for interprocess communication (or virtual memory shenanigans), which atomic orderings are necessary ?</p>
</blockquote>
<p>Ignore the fact that this is interprocess. The orderings are the same as within one process. I mean, it's two threads sharing <em>some</em> of their address space that are synchronizing here.</p>



<a name="162030345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030345">(Mar 29 2019 at 09:27)</a>:</h4>
<blockquote>
<p>unless you use volatile</p>
</blockquote>
<p>yes. not that any formal model would cover this, but volatile should be sufficiently restricted to achieve this.</p>



<a name="162030408"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030408" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030408">(Mar 29 2019 at 09:28)</a>:</h4>
<p>i was hoping that there was a way to do this with a strong memory ordering</p>



<a name="162030582"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030582" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030582">(Mar 29 2019 at 09:30)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> so <span class="user-mention" data-user-id="116155">@Jake Goulding</span> can't do better than using volatile loads / stores, unless, they are ensuring synchronization in some way ?</p>



<a name="162030684"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030684" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030684">(Mar 29 2019 at 09:32)</a>:</h4>
<p>well I think I'd use atomic accesses, I think</p>



<a name="162030789"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030789" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030789">(Mar 29 2019 at 09:34)</a>:</h4>
<p>or do you mean if the same file gets mapped twice?</p>



<a name="162030822"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030822" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030822">(Mar 29 2019 at 09:35)</a>:</h4>
<p>I think at that point I am just out, all I can do is speculate. We are way off in uncharted territory, formally speaking,  and I do not know enough about LLVM to tell if it would exploit pointer comparisons assuming the virtual memory mapping is injective.</p>



<a name="162030882"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030882" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> gnzlbg <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030882">(Mar 29 2019 at 09:36)</a>:</h4>
<p>i mean mapping a file once, but another process (or thread can also map it), and then two processes (or threads) modify it concurrently</p>



<a name="162030885"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap%20concerns/near/162030885" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/136281-t-lang/wg-unsafe-code-guidelines/topic/mmap.20concerns.html#162030885">(Mar 29 2019 at 09:36)</a>:</h4>
<p>also I'm afraid I'll have to leave now, got a lot of stuff to do. I'll hopefully be back online in the evening.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>